Papers with prompt tuning method
Prompts Can Play Lottery Tickets Well: Achieving Lifelong Information Extraction via Lottery Prompt Tuning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on information extraction tasks focuses on one specific task, but in real-world scenarios, new data of different IE tasks and domains come in a stream over time. |
| Approach: | They propose a parameter- and deployment-efficient prompt tuning method to evaluate the UIE system under a “lifelong learning” setting. |
| Outcome: | The proposed method is able to learn new tasks without forgetting old ones and expand knowledge and functionalities without retraining the whole system. |
Towards Alleviating the Object Bias in Prompt Tuning-based Factual Knowledge Extraction (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to optimize prompts for factual knowledge extraction are undesirable object bias. |
| Approach: | They propose a prompt tuning method that reduces object bias and improves factual knowledge extraction. |
| Outcome: | The proposed method reduces object bias and improves accuracy of factual knowledge extraction. |
CodePrompt: Task-Agnostic Prefix Tuning for Program and Language Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Prompt-tuning methods have been used to solve inefficient parameter update and storage issues in Natural Language Generation tasks. |
| Approach: | They propose a task-agnostic prompt tuning method that reflects the traits of PLM for program language. |
| Outcome: | The proposed method is effective in three PLG tasks, not only in the full-data setting but also in the low-resource setting and cross-domain setting. |
StablePrompt : Automatic Prompt Tuning using Reinforcement Learning for Large Language Model (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have made it difficult to find appropriate prompts for tasks with multiple input-output formats. |
| Approach: | They propose a prompt tuning method based on reinforcement learning (RL) they propose an anchor model and an extension for generating input-dependent prompts. |
| Outcome: | The proposed method outperforms existing methods on a variety of tasks and achieves State-of-the-art performance across diverse types and sizes of LLMs. |
IAPT: Instance-Aware Prompt Tuning for Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for prompt tuning require many soft tokens to guarantee performance . large language models still require a large amount of GPU memory and computations to fine-tune . |
| Approach: | They propose to use a parameter-efficient soft prompt generator to generate idiosyncratic soft prompts for each input instruction. |
| Outcome: | The proposed method outperforms the baselines with comparable tunable parameters and is more efficient than LoRA under the single-backbone multi-tenant setting. |
KnowVrDU: A Unified Knowledge-aware Prompt-Tuning Framework for Visually-rich Document Understanding (2024.lrec-main)
Copied to clipboard
Yunqi Zhang, Yubo Chen, Jingzhe Zhu, Jinyu Xu, Shuai Yang, Zhaoliang Wu, Liang Huang, Yongfeng Huang, Shuai Chen
| Challenge: | Existing methods for integrating layout and image features into pre-training language models are not suitable for few-shot settings. |
| Approach: | They propose to reformulate VrDU tasks into a single question-answering format with task-specific prompts and train the pre-trained model with the parameter-efficient prompt tuning method. |
| Outcome: | The proposed framework can be used in few-shot settings and reduces data requirements. |
SMoP: Towards Efficient and Effective Prompt Tuning with Sparse Mixture-of-Prompts (2023.emnlp-main)
Copied to clipboard
| Challenge: | Prompt tuning has emerged as a successful parameter-efficient alternative to the full fine-tuning of language models. |
| Approach: | They propose a prompt tuning method that utilizes short soft prompts for efficient training and inference while maintaining performance gains typically induced by longer soft prompt. |
| Outcome: | The proposed method outperforms baseline methods while preserving memory usage. |